Graphical user interface for control of a home entertainment system

ABSTRACT

An entertainment system has a personal computer as the heart of the system with a large screen VGA quality monitor as the display of choice. The system has digital satellite broadcast reception, decompression and display capability with multiple radio frequency remote control devices which transmit self identifying signals and have power adjustment capabilities. These capabilities are used to provide context sensitive groups of keys which may be defined to affect only selected applications running in a windowing environment. In addition, the remote control devices combine television and VCR controls with standard personal computer keyboard controls. A keyboard remote also integrates a touchpad which is food contamination resistant and may also be used for user verification. Included in the system is the ability to recognize verbal communications in video signals and maintain a database of such text which is searchable to help identify desired programming in real time.

FIELD OF THE INVENTION

The present invention relates to an interface for a multiple user homeentertainment system, and in particular to the user interface forcontrolling operation of the system.

BACKGROUND OF THE INVENTION

The consumer electronics industry has created many stand alone productsfor specific functions, such as television viewing, video recording andplayback, broadband video receivers, playing recorded music andbroadcast music. Some devices combine functions, such as the combinationTV/VCR, and the audio cassette/AM/FM receiver to name a couple. Onedirection that consumers are moving is toward larger televisions locatedwithin a family room or living room with accompanying high qualitystereo. Multiple components are required, each providing separatefunctions. A large screen television based on a 19 inch to 40 inchpicture tube, or 46 inch to 60 inch projection system is used and viewedfrom a distance of two to five meters. The television includes a tunerfor receiving and decoding National Television Systems Committee (NTSC)signals, infrared receiver circuitry for a remote control, and in manycases stereo and surround sound integrated into it, making it a veryexpensive device. Further, the consumer likely has a videorecorder/player, and perhaps a cable box/set top box to receive cable orsatellite transmission which may also include a video tuner and otherelectronics to handle modulated, compressed and encrypted video signals.In addition, a consumer is also likely to have a separate stereo systemcomplete with CD player, tuner and other audio attachments, such asspeakers. This duplicates much of the functionality of the televisionsystem and adds to the cost of a home entertainment center.

Most consumer electronic devices come with remote control devices, whichas in the case of the television above, require sensing circuitry toreceive and process the signals from the remotes. Such controls aretypically based on IR signals which can be interrupted by someonewalking in front of it, and are not able to be used in a different roomfrom the receiver. While a few have on-screen programming functions, andthere are special remote control devices which can be programmed tocontrol multiple consumer electronic devices found in a homeentertainment center, there is little consistency between such controls.In addition, there is no good way to use different remote control unitsto control different programs generating the information displayed invarious windows on the screen. This leads to confusion of the consumer,and the classic case of the blinking “12:00” as consumers becomefrustrated trying to master all the protocols required to appropriatelycontrol their electronic devices. Trying to program a VCR to record aprogram in the future can also be quite difficult. The expense of theadditional circuitry in all the devices to accomplish these functions isborne by the consumer.

Multimedia based personal computers today are configured with CD Romdrives, and speakers as well as graphics drivers for displaying graphicson a monitor attached to the computer. CD Rom drives are capable of bothreading data, such as computer programs, and reading audio informationsuch as music which is output from the attached speakers. Multimediatitles for running on a PC usually are distributed on CD Rom, andinvolve such things as animated encyclopedias and other books, as wellas games that may incorporate video clips which can be shown on a PCdisplay. More and more of the video information on such CDs iscompressed in accordance with Motion Picture Experts Group (MPEG)standards and requires commercially available software or circuitry todecompress it and process it for display. The display signal istypically of VGA quality. Some add on products for PCs even provide aconnection to video feeds from multiple sources for playing in a windowon the monitor screen. Typically, the computer has a video graphicsadapter (VGA or SVGA) card which processes all the information to bedisplayed on a monitor and the monitor itself is basically a picturetube that shows only what it is sent with very little processing.However, most PC displays are small, and not suitable for viewing bymultiple people at the same time. It often happens that when a familygets a new program such as a game, animated book or educational game,everyone wants to see it being used for the first time, and they huddlearound a small display and vie for positions. In addition, there is nogood way for multiple users to interact on a single computer. Forviewing video feeds from cable or satellite, large screen consumertelevision sets are most commonly used in the home entertainment center.

As can be seen, there is great duplication of function between theconsumer electronics and personal computers, both of which are more andmore likely to be found in a family room or great room of a home. Thisduplication of function leads to much more money being spent to fullyoutfit the home entertainment center and provide additional functions.When one component fails, since it has duplicate function, it isexpensive to replace. One system that tried to solve some of the aboveproblems is shown in U.S. Pat. No. 5,192,999 to Graczyk et al. Thatsystem has a television circuit and an audio circuit within a personalcomputer, both of which are controlled by a remote control device. Thetelevision circuit is used to receive common cable or broadcast videosignals in NTSC format, which inherently have a lower quality than videosignals currently broadcast by digital broadcasting satellite. NTSCformat signals are interlaced, which means that every other line isrefreshed during each scan of the picture tube. For example, odd lineswould be refreshed during a first scan, and even lines during a secondscan. Since there are 60 scans per second, odd lines are refreshed 30times per second, and even lines are refreshed 30 times per second. VGAmonitors refresh at least 60 times per second, providing a more coherentspacial and temporal image. The NTSC signal is a lower bandwidth signalthan digital MPEG, which has a much higher bandwidth and allowsproductive use of even higher resolution monitors. While Graczyk et al.does convert the NTSC signals to VGA format for display by a dataquality analog monitor, the signal quality is limited because of thelower bandwidth transmission. NTSC signals have a great effect on thetype of text that can be displayed. With an interlaced display, the textdisplayed in normal fonts appears to jump as alternate lines arerefreshed. There have been several attempts to design fonts thatminimize this jump effect, but none have worked well. A VGA display,refreshing each line with every scan of the tube, does not have thisproblem, and provides a much sharper and readable image for text.

The monitor described by Graczyk is shown as a standard PC analog VGAmonitor, and reference is commonly made to a single user. Such monitorsare fairly small, having a maximum viewing area of 17.5 inches (44.5 cm)at the high end of the PC market. They are not nearly suitable forviewing in a home entertainment environment. They are designed for closeviewing, having pixels very close together. Current home entertainmentsystems are much larger and expensive due to all the other circuitrythey have as described above.

One satellite broadcast system is that provided by DirecTV, a unit of GMHughes Electronics. Direct Broadcast Satellites, “DBS” provide more than150 channels of high-quality MPEG based video, sound and data to 18 inch(45.7 cm) receiving antennas. Rights to make subscriber terminals arelicensed by DirectTV, and several other companies contributingtechnology. News Datacom Corporation provides encryption and securityfor the DBS system, providing decryption keys, software and an accesscard for each subscriber terminal. Thomson Consumer Electronics providesconsumer subscriber terminals in the form of set top boxes calledintegrated receiver/decoders “IRDs” which convert the high-qualitytelevision signals into NTSC for showing on a standard television. EachIRD demodulates, decodes, decrypts and outputs video and data, which isdisplayed on a subscriber television. Error correction, decompressionand demultiplexing to separate out video data from other data is alsoprovided in the IRD. While S Video output is provided, there is noprovision for monitor/VGA quality output capability. Other sources ofMPEG based video include optical fiber based cable systems, compactdisk, video clips available on the Internet network, both old andproposed HDCD (high-definition compact disk) formats, and other existingand proposed satellite, digital cable, and asynchronous-transfer mode(“ATM”)-based systems as well as wireless digital broadcasts.

SUMMARY OF THE INVENTION

A multipurpose computer system is provided with circuitry to controlconsumer electronics, such as a large monitor or television for groupinteraction and of text, graphics and video in a home entertainmentenvironment. The circuitry provides audio and video tuning capabilityfor display of received high quality video signals on the monitorwithout an intermediate conversion to a lower quality NTSC format. Thecircuitry decodes the received video signals and converts them to VGAformat which provides images of higher quality than NTSC. The monitor iscapable of directly displaying VGA signals, and is not required to havea television tuner circuit, remote control, or audio circuitry. Thissignificantly reduces the cost of the monitor and improves the qualityof display as opposed to typical large screen televisions which containcomplex tuning circuitry.

In one embodiment, the high quality video signals are received fromsatellites broadcasting digital video signals, digital cable signals andother wireless digital broadcasts. One example is direct broadcastingsatellite “DBS” signals having a quality of video signal higher thanthat provided by NTSC broadcasts. The DBS signals are MPEG-compressedvideo, audio and data. The DBS signals are received by a video receivercircuit board which is compatible with a standard personal computerperipheral component interconnect (PCI) bus and fits within the chassisof a personal computer. The video receiver cards have panel connectorsfor receiving coax cable from an antenna which directly receives the DBSsignals. The card has functional blocks comprising a satellite tuner,digital demodulator, forward error correction, conditional access anddecryption/demultiplexing. The demultiplexor provides the capability ofreceiving data packets which range from information on televisionprograms being broadcast, to computer programs for downloading intocomputer memory.

The video receiver card outputs digital video, audio and data streamsonto the peripheral component interconnect “PCI” bus where it can beaccessed by the computer main processor for manipulation and storage.When stored in a personal computer memory, the program information maybe organized in a commercially available database format. This permitsthe use of database functions to be applied to the data. Rather thanbeing stuck viewing the program information as provided by thebroadcaster, the data immediately becomes manipulable by databasecommands, allowing queries of program information. Searches are used toshow programs having certain actors or subject matter, and even haveprograms meeting the query automatically recorded. One use entails auser designating a series to be recorded, and the database keeps trackof what has been recorded or already viewed, avoiding duplicaterecording, as well as providing easy access to the recorded programs. Ingeneral, by capturing data along with video and audio in a form which apersonal computer can process, the possibilities for computerapplications are endless. Interaction with movies, classrooms, otherplayers of games such as golf and a host of other possibilities becomesclear.

Further embodiments of the system include a settop box version, whereall the circuitry is integrated into one or two cards in a box designedto sit on top of a television set having VGA input. In another version,all the circuitry is included inside of the television chassis.

In one embodiment of the invention, closed caption information providedin video signals is captured, and stored in the database for searching.This provides the ability to instantly find current programmingdiscussing events that a user is interested in. In a further embodiment,speech recognition circuitry is used to convert speech to text orcommands for similar searching capability. Given current speechrecognition capabilities, not all words may be recognized. Only thosethat are recognized are stored in the database or used as a command.Much of the information so captured is not relevant in a certain amountof time, so a data retention mechanism is used to identify old data anddelete it from the database once the database has exceeded its allocatedresource level. A standard FIFO algorithm based on the time expiredsince the programs described have been shown is employed. Furtheralgorithms are user selectable to relevancy rank data in accordance withuser preference.

A video graphics adapter (VGA) card also coupled to the PCI bus convertsthe digital data stream into VGA and video signals for display on themonitor. In addition to being a home entertainment system, a fullyfunctional computer system forms the heart of the entertainment system.Instead of buying all the individual consumer electronics parts, such asa large screen television, settop box for receiving broadband video,audio amplifier/receiver, CD player, universal remote control, videogame machine, answering machine and fax, and a personal computer as someconsumers do today, they need only buy the home entertainment system ofthe present invention. The cost is about the same as that for theindividual traditional consumer electronics parts, making the personalcomputer essentially free.

Support for remote control of both the personal computer and the monitorfunctions is also provided in the personal computer as well as standardPC VGA graphic display functions normally associated with personalcomputers. Additional functions also become available based on theintegration of devices and digitized data, video and audio. In thismanner, no audio, remote control or channel tuning electronics isrequired in the display, which results in a much lower price for a largemonitor suitable for home entertainment. Both a handheld remote controlhaving standard television and in one embodiment, video cassetterecorder controls, and a full function remote keyboard having similarstandard television and video cassette recorder controls are provided.Both transmit key signals identifying the key pressed, as well as asignal identifying the source of the key signal. The key signals arepreferably RF signals typically in the megahertz range, but can also beIR or other suitable form of radiation. RF signals have the advantageover infrared “IR” signals in that they are not interrupted by someonewalking between the remote and the receiver. Light source interferenceand jamming of other IR receivers is minimized by using RF signals. Itmay also be used in a different room from the receiver.

The personal computer contains suitable receiving circuitry, whichprovides indications of the keys being pressed, with the identity of theremote control device sending the signal. In one embodiment, an industrystandard architecture (ISA) board or module is plugged into both thekeyboard and mouse ports. Another bus, such as a serial, RS232 ormicrochannel bus may also be used. The board contains RF receivercircuitry which receives the RF signals, decodes them and routes them tothe appropriate port for processing. A receiving module is programmed todistinguish between the sources of the remote control transmittedsignals, and directs keystrokes to the keyboard port, and mousemovements to the mouse port. The signals at the two ports then controlwhich program the remote key signals affect. If the handheld remote isactivated, the key signals usually would control what channel is beingdisplayed, or cause a recorded program to fast forward, play or reverse.It could however be selected to move to the next image or otherwisecontrol a program designed to display pictures taken on a digitalcamera. Keyboard keys are more likely to control a computer programapplication such as a word processor, spread sheet or electronic mailprogram. In one embodiment, the television like controls are used tooverride the remote hand held television controls in channel selection.Just as easily, the hand held remote controls paging through electronicmail. Additionally, the hand held remote has a pointing device, such asa trackball, or miniature joystick with normally associated keys forselecting functions identified by the cursor on the display being movedby the pointing device. The keyboard contains an integrated touchpad forperforming similar function, and additionally for transmittingsignatures, providing the ability to ensure that an authorized user isrequesting a transaction. Given the entertainment room environment, atouchpad provides the further benefit of being able to withstand greasypopcorn and chicken fingers which might foul other pointing devices. Ina further embodiment, each remote control device has its own uniquecursor associated with it. In this manner, it is easier for the user ofa particular device to know which program will be affected when theremote is used.

The remote controls are also used to control the data supplied to morethan one display. In one embodiment, a display is located in a den,where it is displaying a financial application, such as a checkbookbalancing program. The personal computer is set up to drive multiplemonitors, and the remote control keyboard is assigned to the monitor inthe den to control the running and display of information from thefinancial application.

The remote control devices are used to control one or more cursorsdisplayed on the monitor as part of a graphical user interface into thehome entertainment system. Cells displayed as boxes containingdescriptive text are associated with video programs and with videoprograms and with computer programs, may be selected to view and/or runthe programs. Several icons, such as pictures, video clips or list boxesindicating functions, like record are used to provide functions bydragging and dropping cells onto the icons. Other icons include remind,buy, picture in picture and view full screen to name a few. In oneembodiment, each remote control controls its own cursor to allowmultiple users to control different portions of the system.

In a further preferred embodiment, the remote control devices areequipped with a user mechanism to reduce the power of the transmittedradio frequency signals depending on their distance from the computer.Circuitry is provided to detect the power level of the signal receivedand provide an indication on the computer screen, or transmit powerlevel signals back to the remote control device indicative of the powerlevel of the signal received. The remote control device has means toadjust the power, either by the user varying the resistance of the poweramplifier, or by digital adjustment of the power amplifier based on thepower level signals transmitted from the computer. The ability tocontrol the power level of the signals helps lengthen the time requiredbetween replacing batteries which power the remotes.

In yet a further embodiment, remote earphones are provided. The personalcomputer has the capability to generate separate sound tracks for eachwindow being displayed on the monitor. It assigns each to one or moresets of earphones and transmits a local FM signal or broadcast IR or RFwhich is received and played by the earphones set to the right channel.The earphones have tuning circuitry that allows them to be set to adesired channel, or they may be fixed to different frequencies, andsoftware is used to set the frequency or frequencies on which the audioassociated with a window is broadcast.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a home entertainment system for processingand displaying high quality video in accordance with the presentinvention.

FIG. 2 is a top view representation of the end of a shell connector forcoupling to a monitor in accordance with the present invention.

FIG. 3 is a block diagram showing major components of a personalcomputer in the home entertainment system of FIG. 1.

FIG. 4 is a block diagram of a tuning circuit in the personal computerof FIG. 3.

FIG. 5 is a block diagram of a video graphics adapter in the personalcomputer of FIG. 3.

FIG. 6 is a block diagram of an audio card in the personal computer ofFIG. 3.

FIG. 7 is a block diagram showing input and output connectors which areprovided in the personal computer of FIG. 3.

FIG. 8 is a block functional diagram showing the functional elements ofa database storing programming information which can be displayed on thesystem of FIG. 1.

FIG. 9a is a top view of a hand held remote control device in oneembodiment of the present invention.

FIG. 9b is a cut away side view of a hand held remote control device inone embodiment of the present invention.

FIG. 9c is a cut away side view of an alternative hand held remotecontrol device in one embodiment of the present invention.

FIG. 10 is a top view of a keyboard remote control device in oneembodiment of the present invention.

FIG. 11a is a high level flow diagram showing how commands from theremote controls of FIGS. 9a-c and 10 are processed.

FIG. 11b is a high level block diagram showing the association of remotecontrol devices to applications.

FIG. 12 is a block diagram showing power adjustment controls for theremote control devices of FIGS. 9a-c and 10.

FIG. 13 is a block diagram representation of a user interface displayedon the screen of the present invention.

FIGS. 14a-f are block diagram representations of a user interface forinteracting with the home entertainment system of the present invention.

FIG. 15 is a block diagram representation of a video-conferencing systembased on the home entertainment system of the present invention.

FIG. 16 is a block diagram representation of an alternative videoconferencing system based on the home entertainment system of thepresent invention.

DESCRIPTION OF THE PREFERRED EMBODIMENT

In the following detailed description of the preferred embodiment,reference is made to the accompanying drawings which form a part hereof,and in which is shown by way of illustration specific preferredembodiments in which the invention may be practiced. These embodimentsare described in sufficient detail to enable those skilled in the art topractice the invention, and it is to be understood that otherembodiments may be utilized and that structural, logical and electricalchanges may be made without departing from the spirit and scope of thepresent inventions. The following detailed description is, therefore,not to be taken in a limiting sense, and the scope of the presentinventions is defined only by the appended claims.

Numbering in the Figures is usually done with the hundreds and thousandsdigits corresponding to the figure number, with the exception that thesame components may appear in multiple figures. Signals and connectionsmay be referred to by the same number or label, and the actual meaningshould be clear from the context of use.

In FIG. 1, a home entertainment system is shown generally at 110.External to the home entertainment system, a satellite 112, which in onepreferred embodiment is a HS601 model, operated by Hughes at a 101degree west longitude geosynchronous orbital location, transmits signalscomprising 150 channels of modulated digital video, audio and datasignals at a frequency of about 12 GHz. The satellite signals arereceived by the home entertainment system 110 by an antenna 114containing a low noise block converter amplifier. The antenna ispreferably about 18 inches in diameter and receives left and right handcircularly polarized signals between 12.2 and 12.7 Ghz. The antennaprovides a “downconverted-spectrum” signal between 950 and 1450 MHz viaa coaxial cable or other suitable communication medium 116 to a systemdevice 118, such as a personal computer or other system or circuitrycapable of processing data. Suitable antennas are already beingmanufactured and sold by RCA Corporation by direct sales and throughnumerous major retail chains such as Radio Shack. The system 118contains circuitry and software to further processes the signals fromthe antenna, generally demodulating and decoding the signal to produce aVGA signal. The VGA signal is provided via a standard VGA compatiblemonitor cable 120 to drive a large screen data quality monitor 122suitable for viewing in a family room or entertainment type roomenvironment. The system 118 provides for user input by means of remotecontrols 124 and 126. Remote control 124 comprises a handheld sizedevice with standard television controls and numeric keypad, and in oneembodiment, VCR controls and a pointing device. It provides RF(radio-frequency) or IR (infrared) control signals received by thesystem 118. Remote control 126 is a full function personal computerkeyboard, with additional standard television and VCR controls, pointingdevice which is preferably in the form of a touchpad, and it alsoprovides RF control signals to the system 118. RF control signals wereselected over IR or hardwired in one embodiment due to the homeentertainment environment. It allows the system to be in a differentroom from the monitor 122, or if in the same room, a clear line of sightis not required. In another embodiment, IR control signals were selectedbecause of the availability of many standard circuitry at low cost.Further detail of the remote control devices is provided below.

The monitor cable 120 is a standard type cable typically used on VGAdisplay devices, and comprises up to fifteen electrical conductors,interfacing with the monitor 122 in a D series shell connector indicatedat 210 in FIG. 2. The fifteen leads, some of which are blank, end inpins in the connector which is a molded over, shielded, triple row, 15position, subminiature D, straight male plug. The leads are labelled andidentified in a table indicated generally at 212 next to therepresentation of shell connector 210. The monitor contains a suitablefemale connector for receiving the male shell connector 210.

The monitor 122 in FIG. 1 is preferably capable of displaying at leastVGA quality data and video. At least 640 by 480 pixels of resolution aredisplayable on the monitor. Alternative monitors of higher quality, suchas SVGA providing an even greater number of pixels are also used in afurther embodiment. Many common graphics cards support multiple suchformats, providing great flexibility. The tube size is 33 inches with adiagonal viewing size of 31 inches. An alternative tube size of 29inches with 27 inch viewing area is lower cost and more suitable for anentry model entertainment system. Monitor 122 preferably contains ananalog tube with an aspect ratio of 4 by 3, supports VGA input, has adot pitch of approximately 0.8 to 0.95 millimeters for the 33 inch tubeand 0.65 to 0.8 millimeters for the 29 inch tube with a misconvergenceof 1 to 1.5 millimeters, a bandwidth of 15 Mhz, a brightness of about 33FL. One CRT which may be used is manufactured by Mitsubishi, having amodel and type number of M79KKZ111X. The above specifications may easilybe modified for larger tube sizes, but are designed to provide optimalviewability from a distance of less than two to about four meters for acombination of data and high quality video. If the viewing distance isdesired to be less, the tube size and dot pitch should be decreased. Ifthe tube size is not decreased, more pixels would need to be displayedfor adequate viewing of data. SVGA monitors providing more lines wouldbe more appropriate. Standard digital monitor controls to controlbrightness, contrast, vertical and horizontal sizing and positioning,on/off (rest/resume) are also provided, with both a user accessiblemanual control panel, and circuitry for receiving control informationfrom personal computer 18 in via monitor cable 120. Further embodimentsof monitor 22 include larger displays of 35 and 40 inches viewable andLCD large projection screen type displays. Both aspect ratios of 4:3 and16:9, commonly referred to as wide screen are supported.

Further detail of the functional blocks of system 118 are shown in FIG.3. A processor 310 resides on a system board containing an industrystandard PCI bus 312. A random access memory 314 is coupled to both theprocessor 310 for direct access, and to the PCI bus 312 for directaccess by other components also coupled to the PCI bus 312.

The other components comprise a receiver 316, video graphics adapter,VGA converter card 318, sound card 320 and modem 322. An RF receiver 324is coupled to standard keyboard and mouse ports, which in turn arecoupled through a standard keyboard/mouse adapter to an ISA bus 326 toprocessor 310. Both keyboard 126 and handheld remote 124 transmit RFsignals identifying the key pressed, as well as a signal identifying thesource of the key signal. The RF signals are typically in the megahertzrange, but can also be IR or other suitable form of radiation. RFsignals have the advantage over infrared “IR” signals in that they arenot interrupted by someone walking between the remote and the receiver.This is especially important during a drag and drop type of activity. Ifthe signal is interrupted for example, a needed document might bedropped on a trash can icon, causing the document to be unintentionallydeleted. Light source interference and jamming of other IR receivers isminimized by using RF signals. It may also be used in a different roomfrom the receiver.

RF receiver 324 receives the RF signals, which provide indications ofthe keys being pressed along with the identity of the remote controldevice sending the signal. RF receiver 324 comprises an ISA board ormodule plugged into both standard keyboard and mouse ports. The boardcontains RF receiver circuitry which receives the RF signals, decodesthem and routes them to the appropriate port for processing. A systemcontroller program running on processor 310 supervises software driverswhich are programmed to distinguish between the sources, and controlwhich program the remote key signals affect. If the handheld remote isactivated, the key signals usually would control what channel is beingdisplayed, or cause a recorded program to fast forward, play or reverse.It could however be selected to move to the next image in a programdesigned to display pictures taken on a digital camera. Keyboard keysare more likely to control a computer program application such as a wordprocessor, spread sheet or electronic mail program. However, thetelevision like controls might be used to override the remote hand heldtelevision controls in channel selection.

Further elements of the preferred embodiment of the present inventioninclude a tape drive 330 for backup of data and storage of videoinformation, such as a movie or other program, a fixed disk drive 332for storing programs and data, a CD ROM drive 334, and a diskette drive336, all as commonly known in the personal computer art. A pair ofspeakers 338 and 340 are also provided. A power supply and associatedcontrol circuitry is indicated at 342, and is coupled to the RF receiver324 to receive signals representative of power on and power off commandsfrom the remote control devices. Power circuitry 342 maintains power tothe RF receiver 324 even when the rest of the system is shut down toensure that the power on signal is received and executed. To avoidaccidental powering off of the system, two consecutive power off signalsmust be received before the system is powered down.

In one preferred embodiment, processor 310 is a 100 mhz Pentiumprocessor, RAM 314 comprises a minimum of 16 megabytes, disk drive 332is a 1.5 gigabyte IDE hard drive, the CD ROM drive 334 is a quad speed,3 disc changer, and the fax/modem is a standard 28.8 k bits-per-second(or “KBAUD”) modem. It should be noted that components in personalcomputers are getting faster, smaller and cheaper with higher capacity.It is easily anticipated that larger memories and faster modems andprocessors will be preferable over the next 20 plus years. In analternative preferred embodiment, a very high speed processor 310 isused, and receiver 316 simply comprises circuitry to transform receivedsignals into a digital format. The remainder of the functions of thereceiver 316 and VGA card 318 are performed by software routines toreceive, tune multiple simultaneous channels, decompress, perform errorchecking and correction and convert the digital signals into digital VGAformat. The VGA card 318 in this embodiment simply comprises a digitalto analog converter.

Receiver 316 is shown in further detail in FIG. 4. A tuner 410 receivesthe direct broadcast signal from antenna 114 via standard coaxial cable116. The broadcast signal is 150 channels of high-quality television,including data, compressed in accordance with motion picture expertgroup, MPEG-1 standards with MPEG-2 expected to be available in the nearfuture. The information is carried on an approximately 12 gigahertzcarrier, and the receiver 316 selects channels from the carrier andperforms analog to digital conversion of the signal. A demodulator 412is coupled to tuner 410 for demodulation of the digital signals. Forwarderror correction is provided via standard Viterbi and Reed-Solomonalgorithms at block 414. The output of block 414 comprises an encrypteddigital signal, in one embodiment encrypted in accordance with dataencryption standards “DES.” While it is not important to preciselyreproduce the video signal, it is much more important to get the datasuch as computer programs and financial information accurately. Theabove error correction algorithms are designed to reduce the error ratefor data to 10⁻¹² errors per byte of data. For video and audio, an errorrate of 10⁻⁸ is acceptable. The error corrected encrypted digital signalis supplied to a decryption and demultiplexing block 416. Block 416 iscoupled to a conditional access card 418 which provides the key fordecrypting the digital signal. Upon separating the multiplexed digitaldecrypted signal, it is provided as digital MPEG conforming signals viaa connector 420 to PCI bus 312.

In operation, when processor 310 executes a command, such as changing achannel, it sends the command over the PCI bus 312 to receiver 316,where tuner 410 tunes in a different channel, and the video signals areprocessed and sent on to VGA card 318 and sound card 320. Once on PCIbus 312, the digital MPEG signal may follow several different courses.In one embodiment, a buffer is allocated from RAM 314 to hold a userselected amount of compressed video data or information comprisingcomputer programs. The same buffer concept may be applied to disk drive332, which is useful for holding an even larger amount of information.Tape drive 330 is preferably an 8 millimeter tape device, and is usefulfor storing entire programs of data or MPEG compressed video/audioinformation. The tradeoffs for each storage device are clear, with thecost of storage per megabyte increasing with the speed with which it maybe retrieved. Hence, while RAM provides very fast access, it is alimited resource, and does not make sense to use for much more then afew seconds of video related data. Also, it is a transitory memory inthat it does not persist if power is removed. It is most useful forbuffering about 10 seconds worth of video information. While it is notpossible to predict how long a block of data will play when convertingto video, 10 seconds is about 15 megabytes of data. The tape drive 330is the cheapest form of data storage, but is not randomly accessible. Inother words, it can take some time for the tape to move to the rightplace before desired data can be retrieved for playing. However, it doesprovide the most cost effective storage for linear playback of moviesand television shows. A disk drive 332 is has an average access time ofabout 10 milliseconds, making it a nice tradeoff between tape and RAMfor buffering portions of video information and allowing the user toselect and quickly play back desired buffered video. It, like tape isalso persistent.

When the MPEG data is buffered, a FIFO type of buffer is used, with newinformation written over the oldest information when the buffer is full.Through use of the television and VCR remote controls on handheld remote124 and keyboard 126, VCR-like instant replay functions are providedfrom the buffered MPEG data. The buffered data is sent back underprocessor 310 control via the PCI bus 312 to the VGA card 318 fordisplay on monitor 122.

The VGA card 318 is now described with reference to FIG. 5. In FIG. 5, acontroller 510 is coupled to the PCI bus to receive MPEG encoded video,and other normal personal computer display information such as graphicsand text. If controller 510 detects MPEG data on PCI bus 312, it routesit to a decoder 512 with associated dynamic random access memory of 2megabytes, DRAM 514. Decoder 512 decodes the MPEG data in accordancewith MPEG standards. A commercially available chip from SGS-ThompsonMicroelectronics, part number STi3520 is one such decompressor chipwhich performs the decompression. DRAM 514 is used as a buffer to assistin the decoding, since large amounts of data are required at one time todecode MPEG data. Audio information from the decoded MPEG data isprovided to a header 516 for transmission to the sound card 320. Thedecoded video signal in YUV color encoding is provided back tocontroller 510 which then places the video information into a dynamicrandom access memory or video random access memory, VRAM 518. Aconverter 520 retrieves the information from VRAM 518 and generatesstandard analog VGA display signals on a cable adapter 522 which mateswith shell 210 in FIG. 2 for display.

Other audio video inputs are provided on VGA card 318, including astandard cable connector at 524 coupled to a video tuner circuit 526.Tuner 526 provides both an audio output on line 528 to header 516, andan audio video signal to a video decoder 530 via line 532. Line 532 isalso coupled to a video multiplexer 540, which selects one of at leasttwo NTSC compatible audio video signals it receives to an audio videoout panel connector 542. Video decoder 530 also receives audio videoinputs from further panel connectors, comprising RCA jack 544, S-videopin 546 and a second RCA jack 548. The video signals provided on theselines are decoded by decoder 530 into YUV video output on line 550,which also receives decoded MPEG signals from decoder 512. Both thesesignals are available for display by VGA display via controller 510without conversion to NTSC. An alternative is to route the signals online 550 to a YUV to NTSC converter 552, which feeds video multiplexer540 for display on a standard television of NTSC signals. However,conversion to NTSC results in an inferior picture being generated fromwhat is received in either S-video or from the antenna 114 asrepresented by the MPEG signals on PCI bus 312. By not converting toNTSC, the entire transmitted image may be displayed. Prior art NTSCconversion lead to a cropping of the image to ensure that the imagepresented was not degraded at the edges. The present invention providesthe ability to scale for provision of the entire transmitted imagewithout quality loss at the edges, since it provides a digital videosignal. No overscan is required as is needed to ensure smooth displayborders for NTSC analog transmissions.

Sound card 320 is shown in further detail in the functional blockdiagram of FIG. 6. Both FM synthesis and wavetable synthesis areprovided at 610 and 612 respectively. Their outputs are mixed ormultiplexed at 614 and provided to a mixer and coder/decoder 616 whichprovides a line out 618 containing SoundBlaster compatible output forconnection to standard speakers if desired. In a further embodiment,surround sound compatible output is provided. Mixer 616 also has amicrophone input line 620, a CD audio digital line in 622 oneembodiment, and a CD audio analog line in 624. A multiplexer 626 alsomultiplexes multiple input lines into mixer 616. A line in 628 and twoaudio/video lines 630 and 632 are multiplexed by multiplexer 626, as aresignals received from header 516 via lines 634 and 636. Line 634 iscoupled through header 516 to tuner 526, providing an audio signal fromchannels on cable connector 524. Header 516 is also coupled to MPEGdecoder 512 to provide MPEG PCM (pulse-code modulation) audio throughdigital to analog converter 640 to line 636. Lines 636 and 634 are alsocoupled through a further multiplexer 642 to provide a line out of VCRcompatible audio on a line 644.

FIG. 7 shows multiple ports available on the back panel of the personalcomputer 118, and on the cards shown in FIGS. 4, 5 and 6. The back panelconnections are shown generally at 710. The audio video inputs compriseRCA jack 544 and audio left and right speaker connectors 630 forreceiving audio video input from a VCR or game machine, S-video pin 546and auxiliary RCA jack 548 for receiving audio video input from a VCR,camcorder or laserdisc and left and right audio connectors 712. Audiovideo out connectors 542 and 644 are available on the back panel 710 forcoupling to a VCR for recording. Sound card 320 contains connectors 618,620 and 628 on the mounting bracket, plus a game port 720. VGA card 318contains the coaxial cable connection 524 and the VGA output port 522.Digital receiver card 316 contains the coaxial cable connector tocoaxial cable 116, and a slot for the conditional access card 418indicated at 720. Slot 720 may be a standard PCM/CIA slot (also referredto as CardBus or PC Card), or any other type of connector desired, suchas one specified by News Datacom, who is providing such digitalsatellite system (DSS) cards for settop boxes. By providing the aboveconnectors and ports on the back side of the personal computer, they areaccessible for easy wiring, but also out of view in the homeentertainment center.

The DSS signal on coax cable 116 also contains data relating totelevision programming scheduling and closed caption information. Insome video broadcast services, the closed caption information iscontained in the vertical blanking interval of video signals. Thisinformation is captured by the receiver 316 and separated out from thevideo and audio signals, and provided on the PCI bus to RAM 324 where itis accessible to processor 310. It can also be stored on any otherstorage device capable of storing text or other digital data. As shownin FIG. 8, the text is stored directly into a database 812 on RAM 324 ordisk drive 332. In one preferred embodiment, database 812 comprises flatfiles of the entire text provided by receiver 316. No organization isapplied to it other than sequential as received. A search engine 814running on processor 310 accepts user queries indicated at 816 in theform of key words. All standard commercial database query functionality,such as proximity searching, query by example, wildcard letters, etc areapplicable. Search engine 814 then applies the query to the database 812and supplies the results back through the VGA card 318 for display. Thedata is preferably formatted as specified by the user, most likely interms of the title of the program, the time that it will be on, and thechannel that it will be on. Further information, such as descriptions ofthe program, and at least partial credits including director and actorsmay also be specified to be provided by the user. Closed captioninformation is stored separately from the programming information andmay be queried separately. Where a separate data channel contains allthe closed caption information for the channels containing videoinformation, it is easy to store the closed caption information for allthe channels. However, if it is required to decode each video MPEGchannel, then only the closed caption information from one channel at atime is stored. An auto surf function cycles through desired channels,picking up portions of closed caption information, which can then alsobe searched. This is very useful when looking for current event typeprogramming. Perhaps a user is interested in a particular place, or whena particular feature on a desirable topic starts. A query to review allthe current and past mentions of the place or feature during liveprogramming can quickly alert the user to broadcasts satisfying thequery. Where closed caption is not supported, the sound track from videois fed into a speech recognition program 820 running on processor 310which recognizes at least a few words from current televisionprogramming, and will improve with time and advancement in linguisticparsing. For now, only the words that are recognized are fed into yetanother partition of the database which can be independently searched.

Various views of the hand held remote control 124 are shown in FIGS. 9a,9 b, and 9 c. A 19 millimeter optical trackball 910 is integrated intothe remote, and functions just as any other pointing device for personalcomputers, generating cursor control signals that are transmitted to thepersonal computer 118. In a further preferred embodiment, the cursorcontrol device is a miniature joystick shown at 911 in FIG. 9C, that isoperated by a finger being placed on top of the stick and pushing in thedirection of desired cursor movement. Selection buttons, or mouseclickers are provided at 912 for the user to press to select functionson the personal computer that the cursor is touching. An additionaltrigger like selection button is provided at 913, which is convenientfor use by an index finger if holding and pointing the remote in anatural position. It is also useful as a pseudo trigger for many games,and for one handed dragging and dropping of icons.

In one embodiment, as shown in FIG. 9C, a microphone 1244, an earphone1242, and an RF transceiver are included in remote control devices 124,providing a cordless-telephone functionality, with PC 118 being the baseunit and remote control 124 being the remote unit. In another suchembodiment, as shown in FIG. 10, a microphone 1244, a speaker 1242, andan RF transceiver are included in remote control device 126, providing acordless-telephone functionality, with PC 118 being the base unit andremote control 126 being the remote unit. Phone calls can be received orinitiated by activating a “phone” function by pressing one or more ofthe control buttons on the phone, such as simultaneously pressingbuttons 922 and 928, which, in one embodiment, toggles function betweenTV and phone, thereby automatically muting the audio of the TV whenphone function is desired.

A power switch is provided at 914 to provide power-on/off functions forthe monitor 122 and resume/rest for personal computer 118. Channelcontrol buttons 916 provide the familiar television/VCR up and downchannel control functions. Volume controls 918 are also provided, as isthe standard number keypad in television remote controls indicatedgenerally at 920. Further provided are mute button 922, channel recall924, FAV (favorite channel or menu) button 926 and TV button 928, whichserves to cycle the display through full screen display of a televisionprogram, to putting it in a window, to hiding it completely. Anadditional function set control button 930 is used to select functionsprovided by an operating system beyond those normally provided today.Much as the “Ctrl” and “Alt” keys are used to select differentfunctions, the function set button 930 is similarly used on the handheld remote 124 to select the functions defined by the operating system.FAV button 926 may be unique to each family member, and comprise alisting of the users favorite television programs, games, computerapplication programs, home pages on internet, or other interfaces intothe electronic world.

Hand held remote 124 also contains RF generating circuitry coupled toall the keys and pointing devices for generating RF signalscorresponding to the buttons pressed for reception by RF circuitry 324.The power of the RF signal is adjustable via a thumbwheel indicated at934, which is coupled to further power adjusting circuitry shown in FIG.12. Power is provided by a standard 9 volt cell 936, or multiple double“A” batteries, accessible via a removable panel 940.

Remote keyboard 126, shown in further detail in FIG. 10, is very similarto a MF II keyboard, except that the numeric key pad has been replacedwith television/VCR like remote control buttons, and a touchpadindicated at 1010. In addition, it contains a pair of function set keys1012 and 1014 for invoking the same functions as button 930 on handheldremote 124. The television/VCR buttons include a power button 1016,volume controls 1018, channel controls 1020, TV/VCR button 1022, mutebutton 1024, a favorite channel/function button 1026 and a channelrecall button 1028. In addition, selection keys 1030 and 1032 areprovided to select function indicated by the cursor as controlled bytouchpad 1010. Touchpad 1010 is integrated into the remote keyboard, andpermits easy manipulation of the cursor by simply moving an object, suchas a pen, stick or finger/finger nail across the pad in the desireddirection. It can be useful for performing signatures to validatetransactions, or restrict access to files. By integrating touchpad 1010directly in the remote keyboard 126, there are no external connections,or sliding drawers to contend with, which could easily become cloggedwith grease, butter, sugar or any of the other messy things people eatwhile being entertained in a family entertainment center. The portion ofthe keyboard containing the television/VCR remote controls is preferablysealed, and impervious to being affected by such foods and drinks as arecommonly found in a family room. Since the keyboard is more likely to beheavily used, a higher number of double “A” batteries are used. Four toeight provide a suitable length of operation. They may be rechargeable,and an external power jack 1042 is provided to allow the keyboard to beplugged into standard electrical utility power supplies.

As with the handheld remote 124, the keyboard remote 126 comprises RFgenerating circuitry 1040 to provide RF signals for reception by RFcircuitry 324 in personal computer 118. RF circuitry 1040 also adds on akeyboard identifier with the signals it transmits, which identifies eachkeystroke as originating from the keyboard. In one embodiment, it is aset of bits that is unique to the remote device. This helps preventinadvertent interference from other owners of similar home entertainmentsystems from inadvertently controlling the users system. In anotherembodiment, the frequency is shifted slightly from that emitted by thehandheld remote, and RF circuitry 324 detects the bits, or the frequencyshift and identifies the keystrokes as originating from either thehandheld remote, or the remote keyboard which is associated with thesystem. In yet a further preferred embodiment, multiple remotes areprovided, each with its own identifying frequency or code, includingjoysticks for controlling games. In this manner, each individual in theroom can be controlling their application or program in a window, or beplaying different parts in a game. Since all remotes would be sendingthe same signals to represent the same functions, the drivers for suchremotes running on processor 310 need not differ. They need only bedesigned to handle multiple different sources of the keystrokes, buttonstrokes, mouse, stick or touchpad signals.

A high level flowchart showing how commands issued from different remotecontrol devices are interpreted by processor 310 to control differentapplications is shown in FIG. 11a. When an application is started in aWindows 3.11 or 95 environment, it is initialized as shown at 1110 to beassociated with a particular remote. The keys or buttons from suchremote are identified in groups of either “keyboard” or “TV” or “game”type keys. Game type keys would be those associated with the stick,trackball or mouse type of pointing devices. Thus, an application couldbe associated with game keys from the handheld remote for one player ofa game, and game keys from the keyboard remote for a second player.Further remotes, or different groups from a remote could be used forfurther players. When RF circuitry 324 receives signals from a remote,it identifies the command, such as what key was pressed, and whichremote device it came from to processor 310 at step 1112. Processor 310then identifies the application in a window to which is should beapplied. If it is a presentation type of command, such as enlarging awindow or opening a window as determined at 1116, the command is routedto the window manager for execution at 1118. If not, it is executed onthe application that the group it is associated with was initialized toat step 1110. In further preferred embodiments, individual keys areassociated with applications, however, at least one group or key isalways associated with the underlying operating system to prevent a userfrom being locked out of other applications. One further use of theabove system is to assign TV/VCR controls to television programmingassociated with a window. In this manner, no cursor need be present inthe window, blocking the television programming in order to performchannel selection. In addition, if someone else is watching a program ina different window, their channel selection controls will only affecttheir window.

A block diagram in FIG. 11b represents tables formed by a programmanager to associate remote control devices and the input devices on theremote control devices to programs. Programs, as used herein refer tocomputer application programs and television programs, both of which arecontrollable by remote control devices. A first program 1122 isassociated with a first remote controller, R1, whose input devices D1and D2 control the first program. D1 and D2 correspond to thealphanumeric keys, such as a computer keyboard keyset, or subsetthereof, or a cursor control device, or even the television control padpreviously discussed. Second, third and fourth programs 1124, 1126, and1128 are also associated with remote control devices, and in someinstances multiple remote control devices. Fourth program 1128 iscontrolled by input device sets D1 and D2 of remote R1, and D1 and D2 ofthe second remote R2. The low level granularity of associating sets ofkeys to applications provides great flexibility for a multi-user homeentertainment system.

In a further embodiment, FM earphone headsets are provided to enableeach person to hear only the sound that is associated with their ownwindow of programming. The sound associated with each program is eitherassigned to one of several standard FM frequencies and broadcast in lowpower, much like that in a drive-in movie theater, and each headset istunable to the frequencies broadcast. In another embodiment, eachheadset is tuned to a different single frequency, and the user selectsthe windows whose sound will be broadcast on which frequencies. The topmost window in such a set of windows will have its audio so broadcast.

Further detail of RF circuitry in the remote control devices is shown inFIG. 12. An input device, comprising the keyboard or hand held remote isindicated at 1210. When a key, touchpad command, trackball etc commandis activated, it is provided to a decoder 1212, which decodes thecommand into a signal representative of the command to be transmitted.It also adds a header and check bits to the signal to be transmitted,indicating a unique identification of the remote. The identification inone embodiment is a digital signal which is unique for each remotecontrol for one system and stored on an EE-PROM, while in a furtherembodiment, the code is an eight bit, or a 16 bit code which results inover 64,000 different combinations, making it unlikely that neighborswithin range will have remotes with codes that will control a user'ssystem. The unique identifier may also be represented by selecting adifferent frequency for each remote control device. The system uses thisunique identification as described above to determine which programshould be affected by the particular command issued from the particularremote that issued it.

The Decoder 1212 provides a decoded signal to an FM signal source andmodulator 1214, which in one embodiment generates a desired RF signal inthe 900 megahertz region of the electromagnetic spectrum. This signal isprovided to a power amplifier 1216, which amplifies the RF signal andprovides it to an antenna 1218 for transmission to the system. The poweramplifier 1216 consumes most of the power in the remote control device.A potentiometer 1222 is provided to reduce the gain of the amplifier sothat it is not always amplifying the signals at the highest power level.Since users will vary the distance they are from the system, there is noneed to always transmit at the maximum power level. If a user is closeto the system, the user may adjust the power level downward by turningthe thumb wheel on the remote in the appropriate direction until thesystem no longer receives the transmissions. In one embodiment, thereceiver in the system detects the signal strength received, andprovides visual feedback on the display to assist the user in settingthe power level to a level where no errors in transmission due to lowpower signals are likely to result. Such signal power level detectorsare well known in the art, and one example is shown in U.S. Pat. No.5,193,210. In a further embodiment, the RF receiver 324 also comprisesan RF transmitter, and provides an RF signal representative of the powerlevel back to the remote, which is received by a receiver antenna 1224.Antenna 1224 is coupled to a receiver/demodulator/decoder 1226 whichdecodes the signal and provides a digital signal representative oftransmitted signal strength back to a power control circuit 1228. Powercontrol circuit 1228 is coupled to the potentiometer 1222 for adjustmentof the gain of power amplifier 1216 based on the signal strengthfeedback from the system. Power control circuitry 1228 is also coupledto the input device, to sense when commands are entered. It is alsocoupled to the other components to control when they are supplied power.During touchpad and trackball operation, power is supplied continuouslyto all the electronics. However, after a predetermined period of time,approximately 1 second, with no further commands being sensed, the powercontrol circuitry 1228 cuts off power to most of the other circuitry,and only turns it back on within milliseconds when activity on the inputdevice 1210 is detected. When other circuitry is powered off, the powercontrol circuit remains active, along with the receiver 1226. The powercontrol circuit 1228 buffers commands until the other circuitry is readyto process the commands. Since it turns back on quickly, there is noperceptible delay by the user.

In a further embodiment, each input device command transmitted isacknowledged by the system with an echo of the command. When noacknowledgement is received, the power control circuitry 1228 instructsthe decoder to send the signal again, while increasing the power leveluntil the system properly acknowledges the command. All other commandsare buffered in the power control circuitry until successfultransmission of the command, whereupon they are transmitted at the newadjusted power level. In one embodiment, the times for rebroadcast arerandomly chosen, and prevented if the receiver 1226 detectstransmissions from another remote control device or the system toprevent interference. This is not done if each remote has its owntransmission frequency, which is set, in one embodiment, by a DIP switch1240.

In yet another embodiment, the remote control devices are used as a“speakerphone,” a hands-free intercom-like connection to the telephonesystem. In conjunction with the modem telephone functions, the RFcircuitry 324 transmits and receives in a manner equivalent to the baseunit of a cordless phone. A speaker 1242 on the remote control device iscoupled to the receiver 1226, which receives RF voice from the RFcircuitry 324, and provides the received sound. A microphone 1244 iscoupled to the decoder 1212 for transmitting sound created by the user.Receiver 1226 and decoder 1212 operate in conjunction as the handset ofa standard cordless phone. In a further embodiment, the earphoneheadsets are used in place of the speakers for privacy. In oneembodiment, a headset jack is provided in the remote control device aspart of the block indicating speaker 1242 and standard headsets may beused.

When the user receives a telephone call, a message appears on thedisplay with the identity of the calling source if caller ID is providedby the telephone service selected. This allows easy interface intoavailable databases to “pull” up further information about the callerstored on the system. A key 931 on the remote is used to answer thecall. The keypad is used to enter numbers, with the modem providingstandard DTMF (dual-tone, multiple-frequency) tones.

The remotes 124 and 126 also contain jacks 942 and 1044 for gamecontroller input. A standard personal computer analog game port isprovided, or in conjunction with the RF transceiver capability of theremote control devices, a bi-directional digital port is provided fromjacks 942 and 1044.

One example of the advantages of having a full function computeroperating in conjunction with broadband video is that it opens up thepower of a personal computer to control the video streams and advanceduser functions. The interface to the broadband video is much more userfriendly. In FIG. 13, one example is shown, with a standard televisionlike schedule indicated generally at 1310. It comprises a timelineacross the top, starting at 8 PM and progressing to midnight. Fourchannels, A, B, C and D are shown, but many more may be available. Otherprograms, such as games and word processors which may be selected instandard double mouse click fashion are also shown. Both the time framesand the number of channels shown may be modified by the user enlargingthe window using standard Windows based navigation techniques. Multiplecells, or programs are shown on each channel, and are represented as asuffix of the channel letter for discussion herein. In actuality,program titles and descriptions appear be inserted in each cell. Byclicking on a cell with the right mouse button, further details appearin an expanded version of the cell.

Several icons are tied to functions controlled by the system. A recordicon 1320 is tied to either a separate video cassette recorder, the tapedrive, or the disk drive to record selected channels. A program cell maybe indicated for recording by dragging the cell with a mouse control anddropping it onto the record icon. Similarly, a reminder may be set bydropping a program cell onto a remind icon 1322. If a pay per viewprogram is desired, one simply drags the program cell, which is perhapsoutlined in green, over on top of a buy icon 1324. The buy icon is alsoused when viewing a home shopping channel, or a catalog in a similarmanner. To view a primary program, a user either double clicks on acell, or drags a cell to a view icon 1326 and drops it there. To place aprogram into a picture in picture format, a user drags a cell onto a PIPicon 1328. The picture in picture window may then be resized like anyother window, or moved to another area of the display. All the icons mayalso be arranged in an L-shape surrounding a primary channel beingviewed so that the icons do not interfere with the video images beingdisplayed from the primary channel.

In one embodiment, each remote control device controls a differentcursor, shown as R1 and R2 on the screen in FIG. 13. When the cursor isused to select a program, it becomes the primary controller for thatprogram. The keys on the remote are automatically mapped into theprogram selected. The keyboard however, retains control of the tasklist, and is capable of selecting a program and becoming the primarycontrol device for that program to the exclusion of the hand held remotecontrol device.

In FIG. 13, icons E1 and E2 represent remote earphones. The personalcomputer sound card 320 also contains a multi frequency FM transmitter646 with antenna 648, and has the capability to generate separate soundtracks for each window being displayed on the monitor. It assigns eachto one or more sets of earphones and transmits a local FM signal orbroadcast IR or RF which is received and played by the earphones set tothe right channel. The earphones have tuning circuitry that allows themto be set to a desired channel, or they may be fixed to differentfrequencies. The user drags the appropriate icon over to the programhaving the desired sound track, or to the remote that the user iscontrolling and drops it. If dropped on a program, the sound cardtransmits the sound for that application on the frequency associatedwith the earphones which are tuned to that frequency. If it is droppedon a remote control device, the top window associated with that remotehas the sound associated with it transmitted.

A first screen display provided on monitor 122 to users is shown inblock diagram form in FIG. 14a. The display comprises a set of iconswhich are tied to underlying functions in a Microsoft Windows95 or downlevel version operating environment. This front end is specificallydesigned to be a replacement shell for the normal Windows operatingenvironment to make it user friendly to a family. It acts as anapplication manager, allowing the user to launch other applications andutilities from within it. The front end is organized into the followingareas, each of which leads to further menus: Entertainment, InformationServices, Productivity, and Logon/Help/Configuration controls. The frontend utilizes Win32c application program interfaces, and operates both asa shell and as a stand-alone application. It supports drag and drop andis Win95 logo compliant.

Further icons on the first screen comprise a logon icon for bringing upa dialog box asking for the user name and password and a help icon forbringing up a context sensitive help engine with a data file which isrelated to the screen where the help icon was selected. The dialog boxfor the logon contains an option for a default, or family logon whichmay not require a password.

The front end splits the integrated video processing, audio processingand data processing functionality of the entertainment system intodifferent areas of similar functionality as show in the block diagram ofFIG. 14b, comprising an entertainment icon 1410, an information servicesicon 1412 and a personal productivity icon 1414. Each of these icons,when activated by clicking, or from a task list invoke further menus. Anentertainment menu in a window on the monitor is shown in FIG. 14c,comprising selections such as Microsoft Home Theater, CD audio, DigitalVideo, IR Blaster and Games, all of which may be launched from thiswindow. The Games icon provides a user configurable games menu, whichhas the capability of being user aware, provided the user identifiedhimself or herself at login. The person logged on is also noted at thebottom of the window. Each user is then above to have their favoritegames listed when they are logged in. A home icon or control button isactive, and takes the user back to the main menu. Help controls are alsoactive, and take the user to context sensitive system and helpinformation.

The Information Services icon 1412 leads to a further InformationServices Menu as shown in FIG. 14d. Many commercially availableinformation services are launchable, as well as telephony relatedfunctions tied to the modem, such as voice messages, fax viewing andmail readers.

The Personal Productivity icon 1414 leads to a Personal Productivitymenu in FIG. 14e, where personal productivity software is launchable.While shown as a text based list, the applications may also berepresented by icons.

A configuration menu is shown in FIG. 14f and is found through the firstscreen display. Cancel is the active button in this menu due to thepotential to adversely affect the look and operation of theentertainment system. A user can change the front end to essentially acomputer program based interface, change the clock of the system, browsethe secondary storage for other applications which can be dragged anddropped onto different menus, and remove applications from differentmenus.

One embodiment, implemented on a circuit card or cards for providingvideo conferencing via a public switched telephone network is showngenerally at 1510 in FIG. 15. Broken line block 1511 implements anindustry standard, such as the proposed ITU-T (InternationalTelecommunications Union-Telecommunications) standard, and each elementdescribed hereinafter within block 1511 implements the standards thatare identified in the element block. Input/output devices comprising amicrophone 1512 and speaker 1514 are coupled to a signal converter 1516.Converter 1516 converts signals from the microphone to digital signals,and digital signals to analog speakers for driving the speaker 1514.Converter 1516 is coupled to an echo cancelling device 1518 for reducingfeedback between the speaker and microphone. An audio coder/decoder 1520is coupled to both the cancelling device 1518 and converter 1516, and toa delay circuit 1522, which in turn is coupled to amultiplexor/demultiplexor 1524 for processing the audio signals.Multiplexor 1524 is in turn coupled to a modem 1526, which hascapability for both sending and receiving data on a public switchedtelephone network (“PSTN”) indicated at 1528 in either analog (such as amodem connected to an analog “POTS” or plain old telephone system) ordigital (such as a digital ISDN connection) form.

A video camera 1532 (which can be a stand-alone video camera, acommercially available camcorder, or other imaging device) is coupled tosuitable video inputs on back panel 710, and is used to provide a videofeed to a color space convertor/graphics display module 1535, whichprovides a further video signal to a video coder/decoder 1536, which iscoupled to the multiplexor 1524. Video signals received from the cameraare compressed by the codec (coder/decoder) 1536, and then transmittedover the telephone network by modem 1526. Video signals received fromthe network by modem 1526 are demultiplexed, decompressed and turnedinto signals (in one embodiment, into VGA signals) for display on amonitor. In addition to providing the ability to perform videoconferencing with other compatible systems over a telephone network,several data-exchange protocols are supported at a data converter 1540,which is coupled between the multiplexor 1524 and a data storage device1542 containing data generated by a user application. A system controlicon 1544 represents the ability of the operating system of a computer,such as the computer comprising the home entertainment system, tocontrol the elements of the video conferencing circuitry. System control1544 is coupled to a system control block 1546, which implementsstandard H.246 and in turn is coupled to both the multiplexor 1524 andmodem 1526 to provide operator control thereof.

A further embodiment of a video conferencing aspect of the homeentertainment system is shown in FIG. 16 generally at 1610. Commoncomponents of the home entertainment system are numbered the same as inprevious figures. In this implementation, a plain old telephone system(“POTS”) connection 1612 (which is alternatively a digital connectionsuch as ISDN into the PSTN) can also is used for the video, sound anddata signal transmission. Other users to be connected by videoconference are represented at 1614. A camcorder 1616 (which can be astand-alone video camera, a commercially available camcorder, or otherimaging device) is used to provide the audio/video feed via the backpanel inputs 710, where the video and audio are then provided to thesound 320 and converter 318 cards for digitization. The digitizedsignals are then sent through the system bus to the modem 322 fortransmission. Received signals via the modem are sent to the CPU, soundcard, and video card for decompression and/or playing on monitor 122and/or a stereo/speaker combination 1620. Elements indicated with brokenlines, are advanced features which are easily implementable. Theyinclude a voice capability coupled to the sound card 320 and modem 322for providing a separate voice channel when video conferencing is notdesired. A set of amplified speakers are indicated at 1624 which aredriven directly by the sound card 320 with no need for independentamplification. A digital camera with microphone indicated at 1628 iscoupled directly to a digital signal processor (DSP) with mixing anddecoding functions indicated at 1630. The DSP 1630 is coupled directlyto the system bus 312. Finally, a video cassette recorder 1632 iscoupled to the converter card 318 for recording.

While the system has been described in terms of a personal computer, itis easily modified to encompass a settop box version, where all thecircuitry is integrated into one or two cards in a box designed to siton top of a television having VGA input. In another version, all thecircuitry is included inside of the television chassis.

In one embodiment, full multi-media signal sourcing and destinationingof audio/video/digital-data (A/V/D) broadcasts is provided for.Referring to FIG. 1, one embodiment of signal 116 from satellite dish114 provides digital A/V/D signals from such sources as DirectTV orPrimestar. In another such embodiment, signal 116 provides analog A/Vsuch as NTSC antenna signals. In another such embodiment, signal 157from camera 156 provides analog A/V such as NTSC audio/video signals. Infurther embodiments, signal 175 from cable source 174 provides analogand/or digital A/V/D. I further such embodiments, signal 163 from PSTN162 provides data or phone signals such as ISDN or POTS signals. In oneset of such embodiments, computer 118 is programmed to automaticallyrecord analog signals, such as television programming, onto recordablemedia, such as video tape, in VCR 172 coupled to cable 173. In anothersuch set of such embodiments, computer 118 is programmed toautomatically record digital signals, such as digital televisionprogramming or CD-ROM-type audio, onto recordable media, such asrecordable compact disks, in CD jukebox 168 coupled to cable 169. CDjukebox 168 also plays CDs or CDROMS for use elsewhere. In another suchembodiment, signals are sent to stereo-surround sound system 158 foraudio output to one or more speakers 160, and on cable 151 to TV 150. Inone such embodiment, earphones 154 on cable 155 and gamepad 152 on cable153 provide additional input/output through remote control 126. Homenetwork 164 is “smart wiring” used to transmit data and control withinthe home, coupled by cable 165 to computer 118. VideoBlaster 170provides video-signal processing on cable/connector 171. Cables 175,116, 163, 157, 151, 173, 171, 169, 155, and 153 can be wired coupling orwireless (such as RF or IR signals without wires).

It is to be understood that the above description is intended to beillustrative, and not restrictive. Many other embodiments will beapparent to those of skill in the art upon reviewing the abovedescription. The scope of the invention should, therefore, be determinedwith reference to the appended claims, along with the full scope ofequivalents to which such claims are entitled.

What is claimed is:
 1. Apparatus for controlling the material displayedon a home entertainment system, comprising: a cursor control device; adisplay having a screen for viewing by users; a processor for receivingcursor control information from the cursor control device andcontrolling the position of a cursor on the display; at least one cellof video program information associated with a video program, displayedon said display by the processor; a record icon displayed on the displayby the processor; a recording device coupled to the processor forrecording the video program associated with the cell when the cell isdragged and dropped by the cursor control device onto the record icon;and a remind icon displayed on the display by the processor and operableto cause the processor to provide a reminder when a cell is dragged anddropped onto the remind icon prior to the time at which the associatedvideo programming is available for display.
 2. The apparatus of claim 1and further comprising: a buy icon displayed on the display by theprocessor and operable to cause the processor to obtain the materialidentified by a cell that is dragged and dropped onto the buy icon. 3.The apparatus of claim 2 wherein said material is a product offered forsale.
 4. The apparatus of claim 1 and further comprising: at least onesecond type of cell containing links to computer application programs.5. The apparatus of claim 4, wherein a computer application programrepresented by a cell is executed by the processor when it is selectedby the cursor.
 6. The apparatus of claim 1 and further comprising asecond cursor control device represented on the display by a secondcursor, wherein the processor receives cursor control information fromthe second cursor control device and controls the position of the secondcursor on the display.
 7. The apparatus of claim 1 and furthercomprising: a view icon displayed on the display by the processor andoperable to cause the processor to display a video program in fullscreen mode when a cell is dragged and dropped onto the view icon. 8.The apparatus of claim 1 and further comprising: a picture in pictureicon displayed on the display by the processor and operable to cause theprocessor to display a video program in a window located in apredetermined portion of the display screen.
 9. Apparatus forcontrolling the material displayed on a personal computer homeentertainment system, comprising: a plurality of remote control devicesfor providing command signals; a personal computer further comprising: aprocessor; a main memory; a bus connecting the processor to the mainmemory; a display adapter coupled to the bus; a display driven by thedisplay adapter; a circuit coupled to the bus for receiving signals fromthe plurality of remote control devices and decoding the signals todetermine the commands, including cursor control signals, wherein theprocessor receives the cursor control information from the cursorcontrol device and controls the position of a cursor on the display; atleast one cell of video program information associated with a videoprogram, displayed on said display by the processor; a record icondisplayed on the display by the processor; a recording device coupled tothe processor for recording the video program associated with the cellwhen the cell is dragged and dropped by the cursor control device ontothe record icon; and a remind icon displayed on the display by theprocessor and operable to cause the processor to provide a reminder whena cell is dragged and dropped onto the remind icon to the time at whichthe associated video programming is available for display.
 10. Theapparatus of claim 9 and further comprising: a buy icon displayed on thedisplay by the processor and operable to cause the processor to obtainthe material identified by a cell that is dragged and dropped onto thebuy icon.
 11. The apparatus of claim 10 wherein said material is aproduct offered for sale.
 12. The apparatus of claim 9 and furthercomprising: at least one second type of cell containing links tocomputer application programs.
 13. The apparatus of claim 12, wherein acomputer application program represented by a cell is executed by theprocessor when it is selected by the cursor.
 14. The apparatus of claim9 wherein there are multiple cursor control devices, and each cursorcontrol device is represented on the display by a cursor which isidentifiable with its corresponding cursor control device.
 15. Theapparatus of claim 9 and further comprising: a view icon displayed onthe display by the processor and operable to cause the processor todisplay a video program in full screen mode when a cell is dragged anddropped onto the view icon.
 16. The apparatus of claim 9 and furthercomprising: a picture in picture icon displayed on the display by theprocessor and operable to cause the processor to display a video programin a window located in a predetermined portion of the display screen.17. An icon driven interface for controlling application and videoprogramming on a home entertainment system comprising: a first set ofcells associated with video based programs and arranged in a timeordered manner on a display screen; a second set of cells associatedwith computer application programs on a display screen; a first set oficons on the display screen separate from the cells, and associated withfunctions to be performed related to the first set of cells; a cursorcontrollable by a user of the home entertainment system; a cursorcontrol device for moving the cursor, and controlling the cursor toperform dragging and dropping of the cells on said icons to cause thesystem to perform the associated functions, and to select applicationprograms from said second set of icons for execution on the system; anda remind icon displayed on the display by the processor and operable tocause the processor to provide a reminder when a cell is dragged anddropped onto the remind icon prior to the time at which the associatedvideo programming is available for display.